Deep Learning frameworks for Image Quality Assessment

Technology is advancing by the arrival of deep learning and it finds huge application in image processing also. Deep learning itself sufficient to perform over all the statistical methods. As a research work, I implemented image quality assessment techniques using deep learning. Here I proposed two full reference image quality assessment algorithms and two no reference image quality algorithms. Among the two algorithms on each method, one is in a supervised manner and other is in an unsupervised manner. First proposed method is the full reference image quality assessment using autoencoder. Existing literature shows that statistical features of pristine images will get distorted in presence of distortion. It will be more advantageous if algorithm itself learns the distortion discriminating features. It will be more complex if the feature length is more. So autoencoder is trained using a large number of pristine images. An autoencoder will give the best lower dimensional representation of the input. It is showed that encoded distance features have good distortion discrimination properties. The proposed algorithm delivers competitive performance over standard databases. If we are giving both reference and distorted images to the model and the model learning itself and gives the scores will reduce the load of extracting features and doing post-processing. But model should be capable one for discriminating the features by itself. Second method which I proposed is a full reference and no reference image quality assessment using deep convolutional neural networks. A network is trained in a supervised manner with subjective scores as targets. The algorithm is performing e�ciently for the distortions that are learned while training the model. Last proposed method is a classiffication based no reference image quality assessment. Distortion level in an image may vary from one region to another region. We may not be able to view distortion in some part but it may be present in other parts. A classiffication model is able to tell whether a given input patch is of low quality or high quality. It is shown that aggregate of the patch quality scores is having a high correlation with the subjective scores

No-reference Stereoscopic Image Quality Assessment Using Natural Scene Statistics

We present two contributions in this work: (i) a bivariate generalized Gaussian distribution (BGGD) model for the joint distribution of luminance and disparity subband coefficients of natural stereoscopic scenes and (ii) a no-reference (NR) stereo image quality assessment algorithm based on the BGGD model. We first empirically show that a BGGD accurately models the joint distribution of luminance and disparity subband coefficients. We then show that the model parameters form good discriminatory features for NR quality assessment. Additionally, we rely on the previously established result that luminance and disparity subband coefficients of natural stereo scenes are correlated, and show that correlation also forms a good feature for NR quality assessment. These features are computed for both the left and right luminance-disparity pairs in the stereo image and consolidated into one feature vector per stereo pair. This feature set and the stereo pair׳s difference mean opinion score (DMOS) (labels) are used for supervised learning with a support vector machine (SVM). Support vector regression is used to estimate the perceptual quality of a test stereo image pair. The performance of the algorithm is evaluated over popular databases and shown to be competitive with the state-of-the-art no-reference quality assessment algorithms. Further, the strength of the proposed algorithm is demonstrated by its consistently good performance over both symmetric and asymmetric distortion types. Our algorithm is called Stereo QUality Evaluator (StereoQUE)

Quality Aware Generative Adversarial Networks

Generative Adversarial Networks (GANs) have become a very popular tool for implicitly learning high-dimensional probability distributions. Several improvements have been made to the original GAN formulation to address some of its shortcomings like mode collapse, convergence issues, entanglement, poor visual quality etc. While a significant effort has been directed towards improving the visual quality of images generated by GANs, it is rather surprising that objective image quality metrics have neither been employed as cost functions nor as regularizers in GAN objective functions. In this work, we show how a distance metric that is a variant of the Structural SIMilarity (SSIM) index (a popular full-reference image quality assessment algorithm), and a novel quality aware discriminator gradient penalty function that is inspired by the Natural Image Quality Evaluator (NIQE, a popular no-reference image quality assessment algorithm) can each be used as excellent regularizers for GAN objective functions. Specifically, we demonstrate state-of-the-art performance using the Wasserstein GAN gradient penalty (WGAN-GP) framework over CIFAR-10, STL10 and CelebA datasets.Comment: 10 pages, NeurIPS 201

Siamese Cross-Domain Tracker Design for Seamless Tracking of Targets in RGB and Thermal Videos

Multimodal (RGB and thermal) applications are swiftly gaining importance in the computer vision community with advancements in self-driving cars, robotics, Internet of Things, and surveillance applications. Both the modalities have complementary performance depending on illumination constraints. Hence, a judicious combination of both modalities will result in robust RGBT systems capable of all-day all-weather applications. Several studies have been proposed in the literature for integrating the multimodal sensor data for object tracking applications. Most of the proposed networks try to delineate the information into modality-specific and modality shared features and attempt to exploit the modality shared features in enhancing the modality specific information. In this work, we propose a novel perspective to this problem using a Siamese inspired network architecture. We design a custom Siamese cross-domain tracker architecture and fuse it with a mean shift tracker to drastically reduce the computational complexity. We also propose a constant false alarm rate inspired coasting architecture to cater for real-time track loss scenarios. The proposed method presents a complete and robust solution for object tracking across domains with seamless track handover for all-day all-weather operation. The algorithm is successfully implemented on a Jetson-Nano, the smallest graphics processing unit (GPU) board offered by NVIDIA Corporation

Distributed compressed sensing for photo-acoustic imaging

Photo-Acoustic Tomography (PAT) combines ultrasound resolution and penetration with endogenous optical contrast of tissue. Real-time PAT imaging is limited by the number of parallel data acquisition channels and pulse repetition rate of the laser. Typical photoacoustic signals afford sparse representation. Additionally, PAT transducer configurations exhibit significant intra- and inter- signal correlation. In this work, we formulate photoacoustic signal recovery in the Distributed Compressed Sensing (DCS) framework to exploit this correlation. Reconstruction using the proposed method achieves better image quality than compressed sensing with significantly fewer samples. Through our results, we demonstrate that DCS has the potential to achieve real-time PAT imaging

No- Reference Stereoscopic Image Quality Assessment

We present two contributions in this work 1)a bi variate generalized Gaussian distribution (BGGD) model for the joint distribution of luminance and disparity sub band coefficients of natural Stereoscopic scenes. and 2) a no- reference (NR) stereo image quality assessment algorithm based on the BGGD model

Sparsity based stereoscopic image quality assessment

In this work, we present a full-reference stereo image quality assessment algorithm that is based on the sparse representations of luminance images and depth maps. The primary challenge lies in dealing with the sparsity of disparity maps in conjunction with the sparsity of luminance images. Although analysing the sparsity of images is sufficient to bring out the quality of luminance images, the effectiveness of sparsity in quantifying depth quality is yet to be fully understood. We present a full reference Sparsity-based Quality Assessment of Stereo Images (SQASI) that is aimed at this understanding